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^General  results  in  the  theory  of  time  series  in  m  dimensions  are  obtained, 
thus  providing  a  broad  view  applicable  to  the  various  models.  The  inter¬ 
relationships  among  the  various  types  of  moving  average  models,  MA,  and  auto¬ 
regressive  models,  AR,  and  the  general  ARMA  autoregressive  moving  average  - 

models  are  stressed.  -— *» 


INTRODUCTION 


>f\ 


t/wlj-'fi- <tJL±  1 1 


General  results  in  the  theory  of  time  series  in  m  dimensions  are  determined. 
These  are:  definitions  of  moving  average,  MA,  models;  autoregressive,  AR, 
models;  and  autoregressive  moving  average,  ARMA  models;  stationarity  and 
invertibility ;  the  characteristic  function;  the  autocovariance  and  auto¬ 
correlation  function;  the  power  spectrum;  properties  of  MA,  AR,  and  ARMA 
models;  the  partial  autocorrelation  function;  estimation;  and  a  brief  look  at 
the  multivariate  problem.  Simulation  and  forecasting  are  considered  by 
Aroian  and  Taneja  (1). 


Important  assumptions  are  outlined  below:  The  characteristic  of  an  event  is 

2x,t'  x  ’  <xl'x2 . V'*'  *"»  *  (xrVX2“*2 . Xm”Sn)  •  Meak 

stationarity  is  assumed  in  time  and  space  as  a  minimum  assumption: 


E  (z  0,  o* 

x,t  z 


Eax,t  ■  °'  °a  >  °»  pi,k  *  {E(2x,t2x-£,t-k))/oz' 


^1 '  ^2  '  *  *  ' 


•V* 


All  second  order  moments  exist.  Note  x  may  be  any  coordinate  system;  if  x  is 
dropped,  the  time  series  is  the  usual  one  at  a  point;  if  t  is  dropped,  then 
the  series  is  purely  spatial.  Note  I  may  be  plus  or  minus  but  k  is  plus 
except  in  forecasting. 


MODELS 

The  MA  model  is  defined 
z 


x,t  “  ,kax-n , t-k+ax , t  ' 

n  -  <n1,n2,...,nm),  nI— » 


(2.1) 


m 


ax  t  is  an  i.i.d.  variable  with  mean  zero,  o*  >  0,  Eax  t*x.£  *  0, 

unless  l  ■  k  •  0.  Usually 

-psn<q,  l*k*r,  *xt  -  j_p  *n>X+n  ,t-k+ax,t'  (2‘2) 


an  MA  model  of  temporal  order  r,  spatial  order  pi+qi  in  each  spatial  variable 


liiim.  As  an  example  m  *  r  *  1: 


\ 
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CORRECTIONS 


GENERAL  RESULTS:  TIME  SERIES  IN  M  DIMENSIONS 


1.  P2  U1 


-  a>  <  t  <  00  not  0  <  t,  < 


2.  P3  equation  (3*7)  replace  by. 


.  t  -  Z  «-  ?  X  V*  *5 

d=0  n=-q  k=l 


3,  p3  equation  (  3*11) /  replace  by* 

s.t  -  ♦'1(B*'Vax,t  ■  Xil  1  F*  B",d  **■ 


4.  P4  equation  (3.14)  in  denominator: 


not  f. 


(2.3) 


x,t 


"eiax,t-l~e2ax-l,t-l+ax#t 


replacing  the  i^'s  by  6's, 

The  AR  model  is  defined: 

zx,t  *n*-<*  k^l  ^n,kzx4n,t-k+axrtl 


(2.4) 


temporal  order  r  if  l£k£r,  spatial  order  p^q^,  -qsnip,  in  each  variable,  x^, 
liiim.  An  example  is 

z  .  *  i  +-i4av  +  •  (2.5) 

X|  v  A  X  f  w  1  e  /  V*1  X  |  V 

The  ARMA  model  is  defined: 

zx,t  *  n*-p  k*l  *n ,kzx+n , t-k~  n^-u  k*l  **n  ,kax+n  ,t-k*ax,  t  (2.6) 

of  order  r+s  in  the  temporal  domain,  q+p,  and  u+v  in  each  spatial  variable. 


The  general  case  would  be  -*><n< * ,  )^t<4r.  This  model  is  denoted  by  ARMA 
(r ,s;p,q;u,v) .  An  example  is: 


x,t 


t’l*x,t-l+^2*x-l  ,t-l"eX*x.t-l"e2ax-l,t-l+ax,t 


(2.7) 


All  the  preceding  are  univariate  cases. 


Practical  examples  for  m*l  are  the  flow  characteristics  of  a  river  or  a 
manufacturing  process;  for  m*2,  the  characteristics  of  a  process  in  the  plane 
such  as  storms,  social  or  economic  processes,  geological  and  geographical 
processes  in  time  including  earthquakes;  and  for  m»3  processes  in  space  such 
as  weather  processes,  sunspots,  communications,  satellite  tracking,  and  oil 
exploration . 

PROPERTIES  OF  MA,  AR,  AND  ARMA  MODELS 

An  MA  model  is  stationary,  and  invertible  dependent  on  and  its  represen¬ 
tation  as  an  infinite  AR  model.  Define  9 


®t zx ,  t 


x ,  t— 1 


-1  'Ft 


B  *,  B  2 


B  ,  6. 

V  x 


(6il '  6i2  '  *  *  *  '  6im)  ,6” 


x-£i,t  ' 

ij  mi  ^ 


(3.1) 


Rewrite  (2.2)  in  terms  of  (3.1). 

»Xrt  ■  (1+n»-p  ril  ♦n.k'&’Vt' 

_  n.  n,  n 

where  F*  -  (F  \  F  * . F  m)  . 

x  Xx  x2  xm 

The  characteristic  function  is 

y(Bx-Bt>  -  1+n.?p  Jl  *n,kFxBJ' 

and  ax,t  *  V‘1<Bx'Bt,zxft  * 

m 

For  invertibility  (3.3)  must  converge  on 
where  SQ  *  (Bt : | Bt | <1 ) ,  S-i  »  { Bx  : | Bx| <1) , 

S .  -  (F  : | F  |>1),  Ui*m. 


(3.2) 


(3.3) 

(3.4) 


(3.5) 


Define  ^  ■  E(zx  t2x-t  t-k* '  the  autocovar*anca  between  *x>t  Md  *x-£ft-k* 

Theorem  3.1.  The  autocovariance  function  of  an  MA  process  may  be  found  by 
multiplying  (2.2)  by  t  .  .  .  ,  where  £  *  (£.,..., t  )  ,and  taking  expectations; 
or  better,  the  autocovir Since  function  1 


(3.6) 


and  >K  ^  is  the  coefficient  of  both  B*B*  and  Bx*Bt*  with  *  o*  being  the 

coefficient  of  B°B?  -  1.  Theorem  3  .1  may  be  used  to  find  p.  p  w  for 

x  t  *1 ' *2 •• •  *  9 

MA,  AR,  and  ARMA  models.  The  autocorrelation  function  is  not  symmetric  in  m 


dimensions;  it  is  symmetric  to  the  origin  x  *  0,  t  «  0.  Thus  p. 


-i0 ,  Ok 


°0-k'P£k  ”  p-t-k'  but  ptm  *  to  p-lm'  p0k  *  pk0‘  Por  w*1'  ptk  wiU  have  th* 
same  values  in  the  first  and  third  quadrants,  and  the  second  and  fourth 
quadrants.  For  m*2 ,  four  sets  of  equal  p's  occur.  In  m  variable  similar 
results  hold,  as  reported  in  Perry's  ongoing  Ph.D.  thesis.  An  important  cut¬ 
off  property  for  the  MA  models  is  given  by: 

Theorem  3.2.  The  autocorrelation  function  for  a  finite  MA  model  is  finite. 
Use  theorem  3.1  for  the  proof.  This  cutoff  property  is  an  important  way  of 
determining  where  a  process  is  MA,  AR,  or  ARMA. 

Theorem  3.3.  If  the  conditions  for  invertibility  are  satisfied,  every  finite 
MA  process  in  m  dimensions  may  be  expressed  as  an  infinite  AR  model.  From 
(3.4) 

a  .  ■  (-  f  .  L  *  .FnB*)d  z  .  0<d<»  ,  (3.7) 

x,t  n~-q  k*l  n,k  x  t  x,t 

and  condition  (3.5)  must  be  satisfied. 

Theorem  3.4.  The  power  spectrum.  Let  «  exp-2Tif,  *  exp-2rig^,  in  the 

autocovariance  function,  theorem  3.1,  the  power  spectrum^of  an  MA  process  is: 

p(f,g)  =  2ofy  (exp-27Tif ,  exp-2Trig)  * 

cl 

¥<exp2Trif,  exp27iig)  (3.8) 

*  2o^  |  Y  (exp-2Trif  ,  exp-2rig)  |2 
Oi  |  f  |  <1/2  ,  0i|gjUl/2,  l<j<m. 

Next  AR  processes  are  considered. 

Theorem  3.5.  The  autocorrelation  function  of  an  AP  process  is  found  by 

multiplying  (2.4)  by  z  0  .  .  and  take  expectations.  For  t  *  k  *  0 

x- X. , 


{ 1-  ?  .  £  ,  d>  up  ,  ) 

z  a  n*-p  k*l  *n,k  n,k 


Note  o  >  0,  and 
z 


px,t  “n--p  k*l  *n,k°x+n,t-k 


(3.10) 


Replace  x,t  by  m,n.  Note  p«k  satisfies  the  same  form  as  (2.4)  for  all  (i,k), 
except  i  *  k  «  0.  The  autocorrelation  (cross  correlation)  function  may  be 
plotted  in  m+1  dimensions  and  is  infinite  in  extent.  As  £  or  k  or  both 
approach  the  corresponding  p's  approach  zero,  provided  the  $*s  are  such 
that  the  AR  process  is  stationary.  The  difference  system  needs  some  p  —  to 
start  the  process. 

Theorem  3.6.  If  conditions  for  stationarity  (3.5)  are  satisfied,  every 
finite  AR  process  may  be  represented  by  an  infinite  MA  process. 


Proof:  From  (2.4) 

*x,t  ■ 

0  £  d  <  • 


(£E*n,kBxBt,d  Vt 


(3.11) 


Theorem  3.7.  For  invertibility  of  an  MA  model  |B  |sl#|Bt|sl  in  (3.3)  set 
equal  to  zero  restricting  8n  k(^n  k) .  For  AR  restrictions  on  $n  k  the  same 

method  is  used  in  the  corresponding  characteristic  equation  of  the  AR  model 


•‘W 


(1-n-£q  k*l  ♦n.kBxBt,‘ 


(3.12) 


This  theorem  may  also  be  stated  for  the  MA(AR)  models,  or  in  fact  for  the 
ARMA  model.  The  roots  of  the  characteristic  equation  must  lie  outside  the 
unit  circle  in  each  variable  B.  when  all  other  B's  are  set  to  one.  This 
condition  holds  as  stationarit?  in  m  dimensions,  m+1  variables,  requires 
stationarity  in  every  direction  in  the  m+1  variables. 


Theorem  3.8.  The  power  spectrum  for  an  AR  process  is: 
p (f  ,g)  *  2o*|4>(exp-2nif ,  exp-2irig)  J~2 
0  *  |f|  *  1/2,  0  s  |?j|  *  1/2,  1  <  j  <  m  . 

Note  the  similarity  to  (3.8). 


(3.13) 


Theorem  3.9.  Given  an  AR  process  to  determine  p£  ,  it  is  necessary  to  first 
find  the  corresponding  MA  expansion  of  the  AR  prodess.  Then 


,m  M  n--00  k^O  ^n,k^n+£,k+m 

C"  n»L  klo  <k 


(3.14) 


where  Cc  is  the  covariance  E(z  .  c  ).  This  is  very  important  since 

the  recurrence  relationship  of  p£  given  by  (3.10)  does  not  provide  a  way  of 

finding  all  the  p£  •  An  AR  model  may  be  simulated  given  {$n  If  this  is 

done  a  set  of  estimated  p_  _  are  found.  The  true  values  are  found  from 0 . 14 )  . 

m,n 

THE  PARTIAL  AUTOCORRELATION  FUNCTION 

One  of  the  most  important  questions  that  must  be  faced  is  the  choice  of  m. 
Usually  spatial  considerations  make  clear  the  value  of  m.  Otherwise  choose 
that  m  which  minimizes  but  theoretical  considerations  should  have  the 
greater  weight.  If  m  is*given,  how  is  r  the  order  determined?  For  the  MA 
model  use  the  cutoff  property  of  the  autocorrelation  function.  For  the  AR 
model  the  cutoff  property  is  provided  by  the  partial  autocorrelation  function. 
For  m*0,  it  is  known  that  the  last  coefficient  $  in  an  AR  model  with  terms 
is  a  partial  coefficient  of  correlationrand  more  importantly  $  +. , 
i>  r  are  all  zero.  AR  models  in  m  dimensions  have  a  similar  property. 

For  r*l  in  m  dimensions  m+1  "last”  coefficients  <J>  will  be  partial  coefficients 
of  correlation,  non-zero,  but  all  other  0's  would  be  exactly  zero.  In 
samples  of  n,  n  large  these  other  0's  will  tend  to  be  small  instead  of  being 
exactly  zero  as  in  the  theoretical  model. 

The  definition  for  the  partial  coefficient  of  correlation  for  AR  models  is 
*ij  *  0  *or  t  j*l,2,..,r,  -1  &  $>ir  £  1  and  0^  *  0  for  i  >  m, 

j  >  r.  For  any  AR  models  for  0^,  i  >  m,  i  >  r,  one  may  prove  0 ^  *  0. 


YULE-WALKER  EQUATIONS,  AR  MODELS 

The  Yule-Walker  equations  for  AR  models  in  m  dimensions  will  be  given. 
Suppose 

*x,t  “  Vx,t-l+Vx-l,t-l+Vf  (5 

Multiply  (5.1)  by  the  coefficients  of  0.  and  02,  take  expected  values  and 
obtain  the  Yule*Walker  equations  1  4 

p0i  -  w10 


ell  •  Vl0+*2 


which  solved  for  and 

♦i  ■  (porpupio"<l-p?o> 


I 


*2  “  <oll"p01p10>/(1_e10) *  <5‘ 

This  is  the  usual  least  squares  used  in  regression.  Hence  the  Yule-Walker 
equations  in  m  dimensions  for  any  order  r  may  be  found  by  the  same  method. 

Thus  the  Yule-Walker  equations  will  be  of  the  form  p  *  P  ♦,  P  and  *  are 
column  vectors  and  P  is  the  matrix  of  correlations.  Bypthe  usual  least 
squares  theory  p 

v(«)  *  n"1(l-p1*)P21  >  (5' 

and  if  we  substitute  the  sample  values  r  for  p  then 

n  /  *  \  __  ’1  /  1  „  1  A  \  n”  1 


Vlh  =  n"1(l-r1*)R2 
r1  ■  <roi»rn)  >  R2 


(‘  ri°) 

»ri°  1  '/i  _r  \ 


,  r"1  =  <l-rj0) 


V((f1,«2)  -  n"1<1-roi*l"rll(!’2>R21 

®?x  “  °l2  ’  n_1 (1"r10>*'1 <l“*i"*2-2*i*2r10) ' 


and  P2  =  -r. .  This  method  is  general  and  if  av  *  are  distributed 

0 .  2  10  * 

normally  the  estimates  $  are  asympototically  unbiased,  consistent  and  approx¬ 
imately  the  maximum  likelihood  estimates;  Perry  and  Aroian  (2),  Aroian  and 
Taneja  (1)  have  extended  this  method  to  the  MA  and  ARMA  models. 


ARMA  MODELS 


Denote  the  characteristic  equation  of  the  MA  model  by  0(Bx#Bt)  and  that  of  the 


AR  model  by  ♦(Bx,Bt>,  then 


♦‘VV’x.t 


0(VBt»ax,t 


B  *  (B  ,B  ,  • . .  ,B  ) ,  an  m  dimensional  ARMA  model  of  order  (r , s;p,q;u, v) . 

X  X  <|  X  X 

j.  z  in 

Now  (6.1)  may  be  written  as  an  infinite  MA  model: 

zx,t  -  e«*»V*"X(#x*Wt  '  (6-2) 

or  as  an  infinite  AR  model 

ax,t  -  *‘Bx'Bt)©  '1(Bx>Bt)*x#t  .  <6. 3) 

Both  results  are  important,  particularly  (6.2),  since  it  is  useful  in  finding 
n,  and  in  forecasting  t+£  #£1'^2  >  T^e  ^«*trictions  on  and  6^ 

for  invertibility  and  stationlrity2of  (6.1)  are  exactly  those  of  the  MA  and 
AR  model  jointly  as  given  in  theorem  3.7.  An  example  is  given  for  m«l,  r,  • 
r2  *  2  in  Aroian  and  Taneja  (1) .  The  autocovariance  functions  is  a  combina¬ 
tion  of  those  of  the  MA  and  AR  models.  The  partial  autocorrelation  is  similar 
to  that  of  the  MA  model. 

The  power  spectrum  is: 


p(f,g)  *  2o 


2  |0(exp-2ffif ,exp-2nig) 
a  ♦  (exp-2irif,  exp-2  nig)  ] 


0  s  |f|  i  1/2,  0  *  |g  1  <  1/2,  1  *  1  m. 

xj  3 

The  ARMA  model  may  differ  markedly  from  either  the  MA  or  AR  model. 


MULTIVARIATE  AR  MODELS 


A  brief  description  of  the  definition  of  an  AR  multivariate  model  is  given: 

■5.t  *  *(p'r>*x-i,t-j*ax,t  '  (7a) 

where  x  is  the  vector  x  «  (x^  ,x2  # .  * .  .x^)  ,  4>(p,r)  is  a  matrix  of  fc^'s  of  p 
rows  by  rp  columns,  .  a  column  vector  of  p  components,  zp  .  .  . ,  a  column 

x,z  X“±,Z-J 

vector  of  rp  components,  and  a*  a  column  vector  of  p  components,  a  multi- 

x ,  x. 

variate  AR  (p,m,r)  model.  The  properties  of  the  model,  with  an  example  p*2 , 
m«r»l,  are  given  by  Aroian  (3). 
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